SGI Developer Toolbox 6.1/SGI Developer Toolbox 6.1 - Disc 1.iso/toolbox/documents/audio/audio.apps/dev/mixing

Internet Message Format | 1996-11-11 | 8.6 KB

Path: gazette.engr.sgi.com!candiru!cook
From: [e-mail] (Doug Cook)
Newsgroups: comp.sys.sgi.misc
Subject: Re: [Q] How to merge sounds by multiports ? [long]
Date: 20 Apr 1995 16:16:34 GMT
Organization: Silicon Graphics, Inc.  Mountain View, CA
Lines: 143
Message-ID: <3n61d2$[e-mail]>
References: <3n2rd4$[e-mail]>
NNTP-Posting-Host: candiru.engr.sgi.com

In article <3n2rd4$[e-mail]>, [e-mail] writes:
>    My question is how to play several sounds simultaneously. So I've got to
> take care how to get these sounds together - merging - then send them to
> the physical output port. I tried to use several same 'sproc' , with each
> one, open a logical ALport that is different to the others. So you know, I
> just send sound data to these ALports 'simultaneously', I expect they can
> be mixed automatically by the advanced sound fascility of SGI.  However,
> the result seems it doesn't work. The sounds played are cliped and lost
> their most fidelities.

In general, using multiple ports isn't a great way to do mixing. It
consumes more system resources and it's more difficult to synchronize
the different sounds precisely. (You CAN do sample-accurate
synchronization between multiple ports in the same direction; use the
ALgetframenumber() call).

But the best way is to  mix the sounds together in your application,
and write them into one port. Mixing isn't hard, but you need to choose
the right algorithm.

Here's a little blurb on mixing. Well, perhaps not so little, but I wanted
to explain the different approaches so you could pick one that works well
for your problem.

The simplest mixing algorithm is just to add the sounds that you want to
play together. Suppose you have two sounds, A and B. For each sample of
output, you add together the two corresponding input samples,
    short a, b, mix;

    mix = a + b;

The trick with this, and with all mixing algorithms, is that you must
be careful that the result does not overflow the numeric representation
you are using. In other words, if A and B are both of type "short," and
the result, mix,  is of type "short," you have the potential to
overflow 16 bits. If A is 32767 and B is 1, then (A+B) in a 16-bit
two's complement number is -32768. In effect, this creates a massive
discontinuity in the output, because the signal "wraps around" from 32767
to -32768.

To minimize this problem, you typically need to do your mixing into a
data type with extra dynamic range, limit the result to the appropriate
number of bits, and then stuff it into a smaller representation. The
modified version of the previous pseudo-code is:
    short a,b,mix;
    int result;

    result = a + b;            /* mix won't overflow because it's more bits */
    /*
     * limit the sum before we go from 32->16 bits
     */
    if (result < -32768) {
        result = -32768;
    }
    else if (result > 32767) {
        result = 32767;
    }
    mix = result;            /* put the result into 16 bits */

Limiting will ensure that if your result does go out of range, it will not
create massive discontinuities. Because of this, it's common practice in
mixing algorithms. (Note that if you're mixing a number of signals together,
you don't have to limit on each addition, only on the final sum, unless you're doing
enough 16-bit additions to overflow 32 bits).

However, even though limiting avoids clicks and pops, the sound will still get
distorted if the sum is out of range, because the actual sum cannot be represented.
This is probably what's happening when you open several audio ports and find
that the mixed result "loses fidelity."

To avoid this problem, the mixing algorithm can scale the signals. There are
several approaches to this. The simplest is to scale the sum of the signals.
This preserves the relative amplitudes between the signals, but allows the result
to be scaled to fit in the 16-bit output range. The equivalent version of the
mixing code would be:
    short a,b,mix;
    int result;

    result = a + b;            /* mix won't overflow because it's more bits */
    mix = (result >> 1);        /* divide by 2, put the result into 16 bits */

This approach is guaranteed not to overflow because we're dividing by
2; since we only have 2 16-bit signals, the result can't be more than
2*32767 or less than -2*32768, so after we divide by two, we don't have
to limit the result. If we had 3 signals, we could divide by 3, and so
on. However, this approach has some problems. First, it makes the
sounds substantially quieter. If b = 0, then the output is a/2, meaning
that we hear a much quieter version of a. Second, it loses some
information.  Dividing by 2 essentially means that we go from an n-bit
signal to a (n-1)-bit signal for each of the inputs. We lose a bit of
precision: if the input signal was 101, the output will be 50, which
introduces a quantization error of 0.5, since 101/2 is really 50.5.

The most general approach is to scale the input signals themselves.
This is the "professional" approach,  and the most complicated. I'll
show a version which assumes the inputs are in floating-point, but it
can be done in fixed-point, too (see my note about floating-point and
the MIPS processor).

    float a,b, result;
    float scale_a, scale_b;
    short mix;

    result = scale_a*a + scale_b*b;
    /*
     * limit the sum before we go from FP->16 bits.
     */
    if (result < -32768.0f) {
        result = -32768.0f;
    }
    else if (result > 32767.0f) {
        result = 32767.0f;
    }
    mix = result;        /* convert FP->16 bit integer */

[NOTE on floating-point and the MIPS processor. Contrary to traditional
wisdom about microprocessors, floating-point multiplies tend to be much
faster than integer multiplies on the MIPS processors. Therefore,
signal-processing code often works best in floating-point. The trick
is that converting your integer data to and from floating-point is
costly. However, if you're doing more than one multiply on a given
number, it's often worth the conversion.  Some guidelines:  a) Don't convert
back and forth more than necessary.  b) Use the -mips2 flag on an R4XXX
or higher if you don't need R3000 compatibility; this will make the
FP->int conversion much faster. c) always use multiplies, not divides,
if you can get away with it.]

The trick with this mixing algorithm is picking good values for scale_a and
scale_b. These will depend upon the input signals. You want them as large
as possible such that the sum of the input never overflows.

Some other notes, for the totally performance-minded:

1. Limiting can be done more efficiently if you want to use the MIPSIV
instruction set; it has conditional-move capabilities. This allows you
to do the comparison without branching. Branching is always a performance
hit since it requires the processor pipeline to be refilled.

2. In general, an M-input-channel to N-output-channel mixer is an MxN
matrix multiply. This is often a good way to think about mixing in the
general case. I've gotten extremely good performance out of a
floating-point matrix multiply algorithm for audio mixing.

Gints Klimanis <[e-mail]> adds that another "gotcha" on floating-
point signal processing is the automatic promotion of some floating-point
operands and arguments to double-precision. Double-precision
arithmetic is typically more costly than single-precision, so if you
don't need it, make sure you're not using it. In some cases the
compiler will automatically promote operands to double-precision
"behind your back." Here are some things to look for. Check out the
"-float" option on the C compiler, which instructs the compiler (in
K&R mode) to avoid promoting operands to double-precision. It's also a
very good idea to use function prototypes with functions taking
"float" arguments -- otherwise the arguments will be promoted to
double-precision (even with the -float option). Finally, if a floating-point
constant is to be single-precision, it should be given in the form "0.0f",
i.e. explicitly single-precision, to avoid promotion to double-precision.

    -Doug


Doug Cook, [e-mail]                 Silicon Graphics, Inc.

"Slab and Esther, uncomfortable with each other, stood in front of an easel in
his place, looking at Cheese Danish #35. The cheese danish was a recent
obsession of Slab's. He had taken some time ago to painting in a frenzy these
morning-pastries in every conceivable style, light, and setting. The room was
already littered with Cubist, Fauve, and Surrealist cheese Danishes. 'Monet
spent his declining years at his home in Giverny, painting the water lilies in
the garden pool,' reasoned Slab.'He painted all kinds of water lilies. He liked
water lilies. These are my declining years. I like cheese Danishes, they have
kept me alive now for longer than I can remember. Why not.'" -Thomas Pynchon